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SUBJECT:  Audit  Report  on  Inpatient  Data  Supporting  the  DoD  Military  Retirement 
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We  are  providing  this  report  for  your  information  and  use.  We  conducted  the 
audit  in  response  to  the  Chief  Financial  Officers  Act  of  1990,  as  amended  by  the 
Federal  Financial  Management  Act  of  1994.  Because  this  report  contains  no 
recommendations,  no  written  comments  were  required,  and  none  were  received. 
Therefore,  we  are  publishing  this  report  in  final  form. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  Questions  on  the  audit 
should  be  directed  to  Mr.  Charles  J.  Richardson  at  (703)  604-9582  (DSN  664-9582) 
(crichardson@dodig.osd.mil)  or  Mr.  Walter  R.  Loder  at  (703)  604-9534 
(DSN  664-9534)  (wrloder@dodig.osd.mil).  See  Appendix  E  for  the  report  distribution. 
The  audit  team  members  are  listed  on  the  inside  back  cover. 
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Inpatient  Data  Supporting  the  DoD  Military 
Retirement  Health  Benefits  Liability  Estimate 


Executive  Summary 


Introduction.  The  audit  was  performed  in  support  of  the  Chief  Financial  Officers  Act 
of  1990  as  amended  by  the  Federal  Financial  Management  Act  of  1994.  Militaiy 
retirement  health  benefits  are  post-retirement  benefits  that  DoD  provides  to  mlitary 
retirees  and  other  eligible  beneficiaries  through  the  Civilian  Health  pd  Medical 
Program  of  the  Uniformed  Services  (Purchased  Care)  and  DoD  military  treatment 
facilities.  Approximately  $172  billion  of  the  $223  billion  FY  1998  estimated  military 
retirement  health  benefits  liability  represented  future  outpatient  and  inpatient  medical 
care  that  the  DoD  military  treatment  facilities  are  expected  to  provide  to  eligible 
beneficiaries.  About  $50  billion  of  the  $172  billion  represented  the  inpatient  portion  of 
the  liability  estimate.  The  remaining  $5 1  billion  of  the  liability  was  provided  through 
the  Purchased  Care.  The  $223  billion  unfunded  liability  was  24  percent  of  the 
$948.5  billion  of  liabilities  included  on  the  DoD  Agency-wide  financial  statements  and 
8  percent  of  the  estimated  $2.7  trillion  of  the  Federal  Employee  and  Veterans  Benefits 
Payable  reported  on  the  FY  1998  consolidated  Federal  Government  financial 
statements.  This  audit  is  the  second  audit  in  a  series  of  audits  to  review  the  reliability 
of  data  elements  used  in  the  estimate  of  the  military  retirement  health  benefits  liability. 
The  first  audit  discussed  the  reliability  of  outpatient  visit  data  that  were  used  in  the 
calculation  of  the  DoD  military  retirement  health  benefits  liability  estimate. 

The  Office  of  Management  and  Budget  issued  Directive  M-99-12,  “Assuring  the  Year 
2000  Readiness  of  High  Impact  Federal  Programs,”  on  March  26,  1999.  The  Office  of 
Management  and  Budget  guidance  directs  DoD  to  report  on  two  areas;  retiree  annuitant 
pay  and  military  hospitals.  The  inpatient  workload  data  discussed  in  this  report  are 
captured  in  the  Composite  Health  Care  System  operated  by  DoD  hospitals  and  clinics. 
The  Deputy  Secretary  of  Defense  reported  to  the  Director,  Office  of  Management  and 
Budget,  that  “100  percent  of  DoD  mission-critical  systems  will  be  compliant  by 
December  1999.” 

Objectives.  The  overall  audit  objective  was  to  assess  the  reliability  and  completeness 
of  the  data  used  to  calculate  the  DoD  military  retirement  health  benefits  liability. 
Specifically,  we  reviewed  the  inpatient  worldoad  data  contained  in  the  Biometrics 
database  at  Fort  Detrick,  Maryland,  for  reliability  and  completpess.  Additionally,  we 
reviewed  the  management  controls  as  they  related  to  the  objective. 


Results.  The  inpatient  medical  records  coding  at  two  military  treatment  facilities  was 
generally  reliable.  Expert  coders  performed  medical  record  coding  reviews  on 
75  records  from  the  2  military  treatment  facilities.  The  medical  record  coding  review 
showed  that  8  percent  of  the  inpatient  medical  records  at  the  two  facilities  required 
changes  in  either  the  diagnostic  related  group  (DRG)  codes  or  the  associated  case 
weights,  or  both.  We  determined  that  the  coding  changes  did  not  significantly  affect 
the  quality  of  the  inpatient  workload  data  and  the  military  retirement  health  benefits 
liability.  In  response  to  Inspector  General,  DoD,  Report  No.  99-127,  “Data  Supporting 
the  FY  1998  DoD  Military  Retirement  Health  Benefits  Liability  Estimate,” 

April  7,  1999,  which  reported  on  the  quality  of  outpatient  data,  the  Assistant  Secretary 
of  Defense  (Health  Affairs)  and  the  Office  of  the  Actuary,  DoD,  initiated  efforts  to  start 
a  quality  assurance  program  to  improve  the  accuracy  and  reliability  of  workload  data  in 
the  Composite  Health  Care  System.  The  issues  identified  in  this  report  were  identified 
as  areas  to  be  included  as  part  of  the  DoD  medical  data  comprehensive  quality 
assurance  program.  Because  management  initiated  efforts  to  correct  conditions  noted  in 
the  finding,  this  report  contains  no  recommendations.  See  the  Finding  section  for 
details  of  the  audit  results. 

The  management  controls  that  we  reviewed  were  effective  in  that  no  material 
management  control  weakness  was  identified.  See  Appendix  A  for  details  on  the 
management  control  program. 

Management  Comments.  We  provided  a  draft  of  this  report  on  December  23,  1999. 
Because  this  draft  report  contains  no  recommendations,  written  comments  were  not 
required,  and  none  were  received.  Therefore,  we  are  publishing  this  report  in  final 
form. 


ii 


Table  of  Contents 


Executive  Summary  i 


Introduction 

Background  1 

Objectives  2 

Finding 

Inpatient  Data  Supporting  the  DoD  Military  Retirement  Health  Benefits 

Liability  Estimate  3 

Appendixes 

A.  Audit  Process 

Scope  8 

Methodology  9 

Managenient  Control  Program  10 

B.  Summary  of  Prior  Coverage  1 1 

C.  Diagnostic  Related  Group  Coding  12 

D.  Statistical  Sampling  Methodology  14 

E.  Report  Distribution  18 


Background 


Requirements  for  Financial  Statements.  Public  Law  101-576,  the  “Chief 
Financial  Officers  Act  of  1990,”  as  amended  by  Public  Law  103-356,  the 
“Federal  Financial  Management  Act  of  1994,”  requires  DoD  and  other 
Government  agencies  to  prepare  financial  statements.  The  Inspector  General, 
DoD,  is  responsible  for  the  audit  of  the  DoD  Agency-wide  financial  statements. 
The  General  Accounting  Office  is  responsible  for  the  audit  of  the  consolidated 
Federal  Government  financial  statements. 

Office  of  Management  and  Budget  Form  and  Content  Guidance.  Office  of 
Management  and  Budget  Bulletin  No.  97-01,  “Form  and  Content  of  Agency 
Financial  Statements,”  as  amended  on  November  20,  1998,  provides  general 
and  specific  guidance  to  agencies  on  the  preparation  of  financial  statements. 

The  Bulletin  established  guidance  for  reporting  pensions,  other  retirement 
benefits,  and  other  post-employment  benefits.  DoD  “other  retirement  benefits” 
include  all  retirement  benefits  other  than  pension  plan  benefits.  The  Bulletin 
further  states  that  entities  that  are  responsible  for  accounting  for  “other 
retirement  benefits”  should  calculate  and  report  the  liabilities  and  related 
expenses  in  accordance  with  Statement  of  Federal  Financial  Accounting 
Standard  No.  5,  “Accounting  for  Liabilities  of  the  Federal  Government,” 
September  1995. 

Military  Treatment  Facilities.  FY  1997  was  the  first  year  that  DoD  reported 
the  unfunded  liability  for  the  DoD  military  retirement  health  benefits  on  the 
DoD-wide  consolidated  financial  statements.  For  FY  1998,  DoD  reported 
$223  billion  for  the  military  retirement  health  benefits  liability.  The 
$223  billion  unfunded  liability  was  24  percent  of  the  $948.5  billion  of  liabilities 
included  on  the  DoD  Agency- wide  financial  statements  and  8  percent  of  the 
estimated  $2.7  trillion  of  the  Federal  Employee  and  Veterans  Benefits  Payable 
reported  on  the  FY  1998  consolidated  Federal  Government  financial  statements. 
Approximately  $172  billion  of  the  $223  billion  FY  1998  estimated  militaiy 
retirement  health  benefits  liability  represented  futore  outpatient  and  inpatient 
medical  care  that  the  DoD  military  treatment  facilities  are  expected  to  provide  to 
eligible  beneficiaries.  DoD  military  treatment  facilities  include  hospitals  and 
clinics.  About  $50  billion  of  the  $172  billion  represented  the  inpatient  portion 
of  the  liability  estimate.  The  remaining  $51  billion  of  the  $223  billion  liability 
was  provided  through  the  Civilian  Health  and  Medical  Program  of  the 
Uniformed  Services  (Purchased  Care).  The  Purchased  Care  program  provides 
health  care  through  civilian  medical  providers. 

Armed  Forces  Medical  Care.  Section  1071  of  title  10,  United  States  Code 
Annotated,  Armed  Forces,  “Chapter  55-Medical  and  Dental  Care,”  requires 
DoD  to  provide  a  uniform  program  of  medical  and  dental  care  for  uniformed 
Service  members,  for  certain  former  members  of  those  Services,  and  for 
dependents. 

Inpatient  Workload  Data  Supporting  Liability  Calculation.  The  Office  of 
the  Actuary,  DoD,  relies  on  inpatient  workload  data  reported  within  the 


Biometrics  database  located  at  the  U.S.  Army  Medical  Information  System  and 
Services  Agency,  Fort  Detrick,  Maryland,  to  calculate  the  DoD  military 
treatment  facilities  portion  of  the  estimated  liability.  Inpatient  workload  data  are 
captured  and  processed  by  the  Composite  Health  Care  System  (CHCS)  at  each 
DoD  military  treatment  facility  that  provides  inpatient  care.  The  CHCS  is  a 
comprehensive  medical  information  system  that  DoD  developed  to  provide 
automated  support  to  its  military  treatment  facilities.  On  a  monthly  basis,  the 
inpatient  workload  data  from  the  military  treatment  facilities  are  electronically 
transmitted  to  the  Biometrics  database  at  the  U.S.  Army  Medical  Information 
System  and  Services  Agency.  The  inpatient  workload  data  in  the  CHCS  should 
agree  with  the  Biometrics  database. 

Inpatient  workload  data  are  measured  by  diagnostic  related  group  (DRG)  codes 
and  relative  weighted  products  (RWPs).  RWPs  are  used  synonymously  with 
case  weights.  The  DRG  code  that  is  assigned  to  inpatient  workload  data  is  the 
result  of  an  inpatient  classification  methodology  that  relates  demographic, 
diagnostic,  and  therapeutic  characteristics  of  patients  to  length  of  stay  and  the 
amount  of  resources  consumed.  Coding  of  medical  records  is  accomplished  at 
each  military  treatment  facility  by  coders.  Coding  involves  a  high  degree  of 
judgment  for  certain  aspects  of  coding,  which  can  result  in  different  coding  for 
the  same  record.  The  RWP  is  a  DoD  measure  of  workload  credit  derived  from 
weights  assigned  to  DRG  codes.  Therefore,  the  RWPs  assigned  to  an  inpatient 
case  is  a  measure  of  the  relative  use  of  resources  that  a  patient’s  hospitalization 
consumed  when  compared  with  those  of  other  patients.  The  Office  of  the 
Actuary,  DoD,  employs  a  methodology  that  uses  the  Biometrics  inpatient 
workload  data  to  calculate  an  estimated  average  cost  to  provide  inpatient  care  in 
DoD  military  treatment  facilities  to  eligible  beneficiaries  by  selective  age 
categories. 


Objectives 


The  overall  objective  was  to  assess  the  reliability  and  completeness  of  the  data 
used  to  calculate  the  DoD  military  retirement  health  benefits  liability. 
Specifically,  we  reviewed  the  inpatient  workload  data  contained  in  the 
Biometrics  database  located  at  Fort  Detrick  for  reliability  and  completeness. 
Additionally,  we  reviewed  the  management  controls  as  related  to  the  objective. 
See  Appendix  A  for  a  discussion  of  the  audit  scppe  and  methodology  and  our 
review  of  the  management  control  program,  and  see  Appendix  B  for  a  summary 
of  prior  audit  coverage  related  to  the  audit  objectives. 
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Inpatient  Data  Supporting  the  DoD 
Military  Retirement  Health  Benefits 
Liability  Estimate 

The  inpatient  medical  records  coding  at  two  military  treatment  facilities 
was  generally  reliable.  However,  8  percent  of  the  records  at  the  two 
facilities  required  changes  in  either  the  DRG  codes  or  the  associated  case 
weights,  or  both.  The  degree  of  changes  required  will  not  significantly 
affect  the  quality  of  the  inpatient  worldoad  data  in  the  CHCS  and 
ultimately  the  military  retirement  health  benefit  liability.  Records 
required  changes  in  part,  because  the  military  treatment  facilities  did  not 
have  either  an  independent  quality  assurance  review  of  DRG  codes,  a 
formal  training  program,  or  a  continuing  education  program  for  medical 
record  coders.  We  could  not  determine  whether  the  inpatient  workload 
data  within  the  Biometrics  database  were  complete  because  of  differences 
between  CHCS  and  the  Biometrics  database.  In  addition,  the  Office  of 
the  Actuary,  DoD,  did  not  provide  adequate  audit  trails  in  its  collection 
of  inpatient  workload  data.  However,  our  limited  review  did  not  identify 
a  significant  detrimental  effect  on  the  quality  of  inpatient  workload  data. 
Also,  in  response  to  an  Inspector  General,  DoD,  Report  No.  99-127, 
“Data  Supporting  the  FY  1998  DoD  Military  Retirement  Health  Benefits 
Liability  Estimate,”  April  7,  1999,  on  the  quality  of  the  outpatient  data, 
the  Assistant  Secretary  of  Defense  (Health  Affairs)  and  the  Office  of  the 
Actuary,  DoD,  initiated  efforts  to  start  a  quality  assurance  program  to 
improve  the  accuracy  and  reliability  of  outpatient  and  inpatient  workload 
data  in  CHCS;  therefore,  this  report  contains  no  recommendations. 


Medical  Record  Coding 

We  obtained  the  services  of  medical  record  coders  to  perform  a  coding  review 
of  the  75  inpatient  records  to  test  the  DRG  codes  at  2  military  treatment 
facilities.  The  medical  record  coding  review  showed  Aat  8  percent  of  the 
records  at  two  military  treatment  facilities,  in  the  opinion  of  the  medical  record 
coders,  required  changes  in  either  the  DRG  codes  or  the  associated  case 
weights,  or  both.  The  changes  resulted  in  incorrect  weights  or  codes,  or  both, 
assigned  to  6  of  the  75  records  reviewed,  which  statistically  represent  2,900  of 
36,246  records  at  the  2  facilities.  Details  regarding  the  audit  results  of  DRG 
codes  are  in  Appendix  C. 
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Completeness  of  Biometrics  Data 


We  could  not  determine  whether  the  inpatient  workload  data  within  the 
Biometrics  database  were  complete  because  of  differences  between  CHCS  and 
the  Biometrics  database.  We  compared  the  FY  1997  inpatient  workload  data  in 
the  Biometrics  database  with  similar  data  captured  in  CHCS.  An  inpatient 
discharge  equals  an  inpatient  workload  unit.  Except  for  minor  adjustments,  the 
inpatient  workload  data  captured  in  the  Biometrics  database  should  equal  the 
workload  data  reported  in  CHCS.  Overall,  we  found  a  difference  of  627  items, 
which  equates  to  less  than  1  percent  difference  when  comparing  the  inpatient 
workload  in  the  Biometrics  database  with  the  CHCS  data.  Although  total 
workload  counts  had  no  significant  difference,  the  accuracy  of  the  inpatient 
workload  counts  varied  by  location.  Specifically,  two  of  the  six  military 
treatment  facilities  had  about  4  percent  difference  rates,  while  four  of  the 
facilities  had  less  than  1  percent  difference  rates. 

The  following  table  is  a  comparison  of  the  inpatient  workload  data  reported  in 
the  Biometrics  and  CHCS  databases  at  each  military  treatment  facility. 


Military 

Discrepancies  in  FY  1997  Total  Discharges 

Military  Treatment 
Biometrics  Facility  CHCS  - 

Absolute 

Treatment  Facility  Database 

Patient  Category 

Difference 

Army 

Dewitt 

5,777 

5,566 

211 

Madigan 

14,751 

14,760 

9 

Navy 

Portsmouth 

25,811 

25,823 

12 

San  Diego 

22,894 

22,947 

53 

Air  Force 

Malcolm  Grow 

7,483 

7,818 

335 

David  Grant 

10,435 

10,442 

_7 

Total 

87,151 

87,356 

627 

The  table  shows  that  the  differences  were  small  between  the  FY  1997  inpatient 
workload  data  reported  in  the  Biometrics  database  and  CHCS.  However,  we 
believe  that  reconciling  the  database  with  CHCS  will  help  ensure  the  accuracy 
and  completeness  of  the  inpatient  workload  data. 
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Management  Controls  Over  Inpatient  Workload  Data 


A  commercial  hospital  that  we  visited  conducted  routine  quality  assurance  tests 
of  DRG  codes.  The  commercial  hospital’s  goal  was  to  keep  DRG  co^ng  errors 
at  less  than  5  percent.  The  six  military  treatment  facilities  that  we  visited  did 
not  have  an  independent  quality  assurance  review  of  DRG  codes  used  at  the 
military  treatment  facilities. 

In  addition,  DoD  did  not  have  a  formal  coding  training  program  and  a 
continuing  education  program  to  enhance  the  quality  of  coding.  The  lack  of  a 
formal  coding  training  program  was  illustrated  at  the  David  Gr^t  Air  Force 
Medical  Center  in  a  November  1996  memorandum  addressing  issues  from  a 
comparison  coding  validation  study  at  the  David  Grant  Air  Force  Medical 
Center,  performed  by  the  Forensic  Medical  Advisory  Service  Corporation.  The 
memorandum  states  that  to  improve  the  coding  process,  management  needed  to 
ensure  that  coders  are  more  than  “minimally  qualified.  ”  Coders  from  the  DoD 
military  treatment  facilities  are  not  required  to  be  certified,  to  have  a  degree,  or 
to  be  anything  other  than  “minimally  qualified.  ” 


Improvements  in  Audit  Trail  for  Inpatient  Data 

The  FY  1997  inpatient  workload  data,  collected  by  the  Office  of  the  Actuary, 
DoD,  to  calculate  the  FY  1998  liability  estimate,  did  not  provide  adequate  audit 
trails.  Specifically,  the  Office  of  the  Actuary,  DoD,  did  not  include  the  Defense 
medical  information  system  identifier  as  a  data  element  in  the  inpatient  workload 
database.  The  Defense  medical  information  system  identifier  is  a  unique 
number  that  identifies  the  military  treatment  facility  in  which  the  patient  was 
hospitalized.  In  FY  1997,  DoD  provided  more  than  412,000  episodes  of 
inpatient  care  in  116  DoD  military  treatment  facilities.  The  Defense  medical 
information  system  identifier  is  needed  to  locate  medical  files  that  were  selected 
for  audit  verification.  The  Office  of  the  Actuary,  DoD,  did  not  include  the 
Defense  medical  information  system  identifier  as  part  of  the  inpatient  workload 
data  requirements  because  the  Office  of  the  Actuary,  DoD,  did  not  need  the 
identifier  to  calculate  the  liability  estimate.  We  brought  the  matter  to  the 
attention  of  personnel  in  the  Office  of  the  Assistant  Secretary  of  Defense  (Health 
Affairs)  and  the  Office  of  the  Actuary,  DoD.  The  Office  of  the  Actuary,  DoD, 
took  action  to  include  the  identifier  as  one  of  the  data  elements  in  the  inpatient 
data  collection  for  the  calculation  of  the  liability  estimate  for  FY  1999  and 
beyond. 
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Management  Efforts  to  Improve  Data  Reliability  and 
Completeness 


The  Office  of  the  Assistant  Secretary  of  Defense  (Health  Affairs)  initiated 
actions  to  improve  the  reliability  and  completeness  of  the  outpatient  workload 
data.  The  Under  Secretary  of  Defense  (Comptroller)  developed  a  strategic  plan 
entitled  the  “Department  of  Defense  Implemeiitation  Strategy  for  Auditable 
Financial  Statements.”  As  part  of  the  overall  implementation  strategy,  the 
Office  of  the  Assistant  Secretary  of  Defense  (Health  Affairs)  took  responsibility 
for  a  specific  issue  area,  “C.3.  Liabilities  Issues,  Post-Retirement  Health  Care 
Liabilities,”  to  address  issues  related  to  the  military  retirement  health  benefits 
liability.  Specific  ongoing  efforts  to  correct  the  impediments  to  achieving  a 
favorable  audit  opinion  include  the  following: 

•  revising  procedures  to  ensure  that  all  data  used  in  calculating  the 
liability  are  current; 

•  revising  procedures  to  ensure  that  reasonably  accurate,  reliable, 
timely,  and  complete  data  required  for  estimating  the  liability  are 
captured  and  reported;  and 

•  conducting  quarterly  in-process  reviews  of  the  military  retirement 
health  benefits  liability  with  principals  from  the  General  Accounting 
Office;  the  Office  of  the  Iiispector  General,  DoD;  and  the  Under 
Secretary  of  Defense  (Comptroller). 

In  addition,  the  Office  of  the  Assistant  Secretary  of  Defense  (Health  Affairs) 
established  a  Medical  Expense  Performance  and  Reporting  System  Management 
Improvement  Group  to  review  deficiencies  related  to  the  Expense  Assignment 
System  and  to  develop  corrective  actions.  A  major  goal  of  the  Group  is  to 
develop  procedures  for  reconciling  financial,  workload,  and  labor  hours  to  the 
data  sources.  When  fully  implemented,  the  procedures  should  help  ensure 
adequate  audit  trails  and  improve  the  reliability  of  the  underlying  data  Aat  DoD 
uses  to  calculate  an  estimate  of  the  military  retirement  health  benefits  liability. 
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Conclusion 


In  response  to  an  Inspector  General,  DoD,  Report  No.  99-127,  “Data 
Supporting  the  FY  1998  DoD  Military  Retirement  Health  Benefits  Liability 
Estimate,”  April  7,  1999,  the  Office  of  the  Assistant  Secretary  of  Defense 
(Health  Affairs)  was  addressing  data  quality  issues  toough  the  establishment  of 
a  data  quality  assurance  program.  The  program  objectives  were  designed  to 
provide  reasonable  assurance  that  inpatient  workload  data  are  being  accurately 
recorded  and  reported.  In  addition,  the  quality  assurance  program  was  supposed 
to  include  testing  the  CHCS  inpatient  information  to  ensure  use  of  complete  and 
reliable  inpatient  data  in  the  calculation  of  the  military  retirement  health  benefits 
liability  estimate. 
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Appendix  A.  Audit  Process 


Scope 

Work  Performed.  Our  audit  focused  on  the  review  of  the  FY  1997  inpatient 
workload  data  that  the  Office  of  the  Actuary,  DoD,  used  to  calculate  an  estimate 
of  the  military  retirement  health  benefits  liability.  According  to  the  U.S.  Army 
Medical  Information  System  and  Services  Agency,  the  FY  1997  inpatient 
workload  consisted  of  412,049  inpatient  discharges  that  were  provided  by  116 
DoD  military  treatment  facilities.  We  conducted  our  review  at  the  following  six 
DoD  military  treatment  facilities: 

•  Madigan  Army  Medical  Center,  Tacoma,  Washington; 

•  Dewitt  Army  Community  Hospital,  Fort  Belvoir,  Virginia; 

•  Naval  Medical  Center  San  Diego,  San  Diego,  California; 

•  Naval  Medical  Center  Portsmouth,  Portsmouth,  Virginia; 

•  David  Grant  Air  Force  Medical  Center,  Travis  Air  Force  Base, 
California;  and 

•  Malcolm  Grow  Air  Force  Medical  Center,  Andrews  Air  Force  Base, 
Maryland. 

The  6  DoD  military  treatment  facilities  that  we  visited  reported  87,151  out  of 
412,049  inpatient  discharges  (21  percent)  for  FY  1997.  We  selected  the  six 
DoD  military  treatment  facilities  for  review  to  provide  audit  coverage  of  each  of 
the  Military  Departments  and  because  of  the  volume  of  inpatient  medical  care 
that  the  DoD  military  treatment  facilities  provided  to  eligible  beneficiaries. 

We  reviewed  the  procedures  for  recording  and  reporting  inpatient  workload  data 
in  the  CHCS  and  the  Biometrics  database  located  at  Fort  Detrick,  Maryland. 
Specifically,  we  selected  and  reviewed  patients’  inpatient  medical  treatment 
records  and  standard  inpatient  data  records  to  determine  whether  the  Biometrics 
database  accurately  reported  the  DRGs,  RWPs,  and  their  associated  bed  days. 

For  FY  1998,  DoD  reported  a  $223  billion  unfunded  actuarial  estimate  for  the 
military  retirement  health  benefits  liability  on  the  DoD  and  the  Government- 
wide  consolidated  financial  statements.  Approximately  $172  billion  of  the 
$223  billion  of  the  FY  1998  estimated  military  retirement  health  benefits 
liability  accrued  from  future  inpatient  and  outpatient  medical  care  that  the  DoD 
military  treatment  facilities  are  expected  to  provide  to  eligible  beneficiaries. 

External  DoD  Assistance.  The  Assistant  Secretary  of  Defense  (Health  Affairs) 
provided  2  certified  DoD  medical  record  coders  to  assist  us  in  validating  DRG 
codes  for  75  inpatient  workload  cases  selected  for  review.  One  of  the  coders  is 


8 


a  Registered  Record  Administrator  and  the  other  is  an  Accredited  Record 
Tectmician.  The  American  Health  Information  Management  Association 
certifies  the  coders. 

The  coders  reviewed  and  re-coded  medical  records  for  75  selected  inpatient 
cases  at  the  Naval  Medical  Center  Portsmouth  and  the  David  Grant  Air  Force 
Medical  Center.  The  DoD  coders  used  the  FY  1997  Encoder  Grouper,  which 
was  also  used  to  code  those  selected  medical  records  by  coders  at  the  visited 
facilities.  The  Encoder  Grouper  is  a  DoD  Medical  Records  software  system 
that  automates  the  process  of  entering  diagnosis  and  procedure  codes  onto  a 
patient’s  medical  record.  Encoder  Grouper  software  analyzes  Ae  various 
diagnosis  and  procedures  codes  that  relate  to  the  inpatient  medical  care  and  then 
determines  the  primary  DRG  code. 

DoD-Wide  Corporate-Level  Government  Performance  and  Results  Act 
Coverage.  In  response  to  the  Government  Performance  and  Results  Act,  the 
Secretary  of  Defense  annually  establishes  DoD-wide  corporate-level  goals, 
subordinate  performance  goals,  and  performance  measures.  This  report  pertains 
to  achievement  of  the  following  goal: 

Performance  Goal  2.5;  Improve  DoD  financial  and  information 
management.  (Ol-DoD-2.5) 

DoD  Functional  Area  Reform  Goals.  Most  major  DoD  functional  areas  have 
also  established  performance  improvement  reform  objectives  and  goals.  This 
report  pertains  to  achievement  of  the  following  functional  area  objectives  and 
goals. 

•  Financial  Management  Functional  Area.  Objective:  Reengineer 
DoD  business  practices.  Gk)al:  Improve  data  standardization  of 
finance  and  accounting  data  items.  (FM-4.4) 

•  Financial  Management  Functional  Area.  Objective:  Strengthen 
internal  controls.  Groal:  Improve  compliance  with  the  Federal 
Managers’  Financial  Integrity  Act  of  1982.  (FM-5.3) 

General  Accounting  Office  High-Risk  Area.  The  General  Accounting  Office 
has  identified  several  high-risk  areas  in  DoD.  This  report  provides  coverage  of 
the  Defense  Financial  Management  high-risk  area. 


Methodology 


Use  of  Computer-Processed  Data.  We  used  the  FY  1997  computer-processed 
data  that  the  military  treatment  facilities  used  to  record  and  report  inpatient 
workload  data.  We  did  not  validate  the  reliability  of  the  CHCS  and  the 
Biometrics  databases  because  we  limited  our  use  of  the  data  to  testing 
management  controls  and  to  obtaining  an  understanding  of  the  procedures  that 
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the  Office  of  the  Actuary,  DoD,  used  to  calculate  the  liability.  However,  not 
validating  the  reliability  of  the  CHCS  and  the  Biometrics  database  did  not 
materially  affect  the  results  of  our  audit. 

Statistical  Sampling  Methodolo^.  See  Appendix  D  for  the  statistical 
sampling  methodology  used  in  this  audit. 

Audit  Type,  Dates,  and  Standards.  We  performed  this  financial-related  audit 
from  January  through  November  1999  in  accordance  with  auditing  standards 
issued  by  the  Comptroller  Genera!  of  the  United  States,  as  implemented  by  the 
Inspector  General,  DoD.  We  included  tests  of  management  controls  considered 
necessary. 

Contacts  During  the  Audit.  We  visited  or  contacted  organization  within  DoD. 
Further  details  are  available  on  request. 


Management  Control  Program 


DoD  Directive  5010.38,  “Management  Control  Program,”  August  26,  1996, 
requires  DoD  organizations  to  implement  a  comprehensive  system  of 
management  controls  that  provides  reasonable  assurance  that  programs  are 
operating  as  intended  and  to  evaluate  the  adequacy  of  the  controls. 

Scope  of  Review  of  Management  Control  Programs.  We  reviewed  Ae 
adequacy  of  management  controls  over  procedures  to  ensure  that  inpatient 
workload  data  were  accurately  recorded  and  reported.  We  did  not  assess 
management’s  self-evaluation  of  those  controls. 

Adequacy  of  Management  Controls.  Management  controls  over  the  inpatient 
workload  data  were  adequate  in  that  we  identified  no  material  management 
control  weaknesses. 
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Appendix  B.  Summary  of  Prior  Coverage 


During  the  last  5  years,  the  General  Accounting  Office  issued  two  audit  reports 
and  the  Inspector  General,  DoD,  issued  two  audit  reports  discussing  the  CHCS 
issues  and  the  DoD  military  retirement  health  benefits  liability,  including  the 
quality  of  the  outpatient  workload  data  supporting  the  liability  estimate. 


General  Accounting  Office 

GAO/AIMD-96-39,  “Defense  Achieves  Worldwide  Deployment  of  Composite 
Health  Care  System,”  April  5,  1996. 

GAO/AIMD-94-61,  “Defense’s  Tools  and  Methodology  for  Managing  CHCS 
Performance  Need  Strengthening,”  July  5,  1994. 


Inspector  General 

Inspector  General,  DoD,  Report  No.  99-127,  “Data  Supporting  the  FY  1998 
DoD  Military  Retirement  Health  Benefits  Liability  Estimate,”  April  7,  1999. 

Inspector  General,  DoD,  Report  No.  99-010,  “DoD  Military  Retirement  Health 
Benefits  Liability  for  FY  1997,”  October  13,  1998. 
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Appendix  C.  Diagnostic  Related  Group  Coding 


Medical  record  managers  at  each  DoD  military  treatment  facility  are  responsible 
for  ensuring  the  accuracy  of  each  DRG  code  diat  is  assigned  to  each  inpatient 
medical  case.  The  DRG  code  assigned  to  an  inpatient  medical  case  represents  a 
significant  computational  factor  that  the  Office  of  the  Actuary,  DoD,  relies  on  to 
calculate  the  military  retirement  health  benefits  liability  estimate.  A  DRG  code 
is  an  indicator  of  the  amount  of  resources  that  a  hospital  stay  consumed.  The 
DRG  code  that  is  assigned  to  an  inpatient  case  is  based  on  factors  such  as 
demographics,  specific  medical  diagnosis  and  procedure,  and  therapeutic 
characteristics  of  the  patient.  The  relative  weighted  product  (RWP)  is  a  DoD 
measure  of  workload  credit  derived  from  weights  assigned  to  DRG  codes.  The 
RWP  is  used  synonymously  with  the  case  weight  that  is  reported  on  the  standard 
inpatient  data  record. 

Medical  record  coders  assign  the  DRG  codes  to  the  inpatient  records  at  each  of 
the  1 16  military  treatment  facilities.  We  obtained  the  services  of  two  DoD 
certified  medical  record  coders  to  review  and  re-code  75  inpatient  medical 
records  as  appropriate  from  2  military  treatment  facilities  visited.  The  2 
military  treatment  facilities  had  a  total  population  of  36,246  medical  records  for 
FY  1997  inpatient  discharges.  The  medical  record  coding  review  showed  that  6 
of  the  75  sample  records  from  the  2  facilities  required  changes  in  either  the 
DRG  codes  or  associated  case  weights,  or  both.  For  example,  the  initial  coder 
at  David  Grant  Air  Force  Medical  Center  assigned  an  inpatient  case  with  a  DRG 
code  144,  “Other  Circulatory  System  Diagnoses  With  CC,”  with  a  case  weight 
of  0.7560.  The  re-coder  re-coded  that  same  inpatient  case  with  a  DRG  code 
Oil,  “Nervous  System  Neoplasm  Without  CC,”  with  a  case  weight  of  0.6482. 
The  initial  coder  coded  “Infection,  Central  Line”  as  the  principal  diagnosis  with 
additional  diagnoses  of  “Bacteremia”  and  “Malignant  Neoplasm  of  Brain.”  The 
initial  coder  also  coded  a  procedure  of  “Magnetic  Resonance  Imaging  of  Brain 
and  Brain  Stem.  ”  Documentation  in  the  record  supported  the  “Malignant 
Neoplasm  of  Brain”  as  the  principal  diagnosis  and  did  not  support  the  other  two 
diagnoses  and  procedures.  Additionally,  documentation  in  the  record  supported 
two  procedures  not  coded  by  the  initial  coder,  which  were  “Magnetic  Resonance 
Imaging  of  Spine”  and  “Chemotherapy,”  which  should  be  the  principal 
diagnoses.  Because  the  initial  coder  picked  the  wrong  principal  diagnosis  and 
coded  some  conditions  that  were  not  applicable,  the  re-coder  changed  the 
principal  diagnosis,  which  resulted  in  a  change  of  DRG  code  and  the  associated 
case  weight. 

Similarly,  the  initial  coder  at  the  Naval  Medical  Center  Portsmouth  assigned  an 
inpatient  case  with  a  DRG  code  41,  “Extraocular  Procedures  Except  Orbit  Age 
0-17,”  with  a  case  weight  of  0.7072.  The  re-coder  re-coded  that  same  inpatient 
case  and  re-assigned  a  DRG  code  48,  “Other  Disorders  of  the  Eye  Age  0-17,” 
with  a  case  weight  of  0.3460.  The  initial  coder  coded  a  procedure  that  was  not 
carried  out  based  on  the  review  of  the  record.  Therefore,  the  procedure  code 
was  deleted  and  resulted  in  a  change  of  the  DRG  code  and  the  associated  case 
weight. 
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We  estimate  that  the  occurrence  rate  for  the  changes  at  the  two  military 
treatment  facilities  was  8  percent.  In  other  words,  of  a  total  population  of 
36,246  records  at  the  2  facilities,  2,900  records  would  require  changes  in  either 
the  DRG  codes  or  associated  case  weights,  or  both.  Accurate  coding  of 
inpatient  medical  records  is  essential  in  compiling  correct  inpatient  workload 
information. 

The  importance  of  a  mistake  in  the  assignment  of  the  DRG  code  is  that  the 
mistake  may  result  in  a  change  to  the  assigned  case  weight  or  RWP  and  a 
potential  impact  on  the  liability  estimate.  However,  the  8  percent  error  rate  at 
the  two  militaiy  treatment  facilities  in  our  review  did  not  have  a  significant 
effect  on  the  liability  estimate. 

The  two  military  treatment  facilities  are  not  necessarily  statistically 
representative  of  DoD  military  treatment  facilities  as  a  whole.  They  were 
selected  judgmentally.  The  sample  results  applied  only  to  the  two  military 
treatment  facilities  sampled. 


Appendix  D.  Statistical  Sampling  Methodology 


Sampling  Objectives.  The  purpose  of  the  statistical  sampling  was  to  assess  the 
reliability  and  completeness  of  inpatient  data  used  to  calculate  the  DoD  military 
retirement  health  benefits  liability.  In  particular,  we  tested  consistency  of  data 
and  processes  within  and  across  Defense  health  care  information  systems.  The 
sample  focused  on  a  subset  of  military  medical  facilities  to  test  consistency  of 
reporting  and  recording  key  elements  of  information  on  inpatient  care. 

Statistical  Sampling  Frame.  The  DoD  military  health  care  system  cared  for 
and  discharged  412,049  retiree  or  retiree-dependent  patients  during  FY  1997. 
The  Office  of  the  Actuary,  DoD,  uses  care  ^ta  from  the  fiscal  year  before  the 
one  being  projected  to  have  complete  data  for  the  year  and  also  to  have  time  for 
conducting  the  actuarial  analysis  necessary  for  computing  the  unfunded  liability . 
The  inpatient  care  was  provided  at  116  military  treatment  facilities.  DoD 
reports  inpatient  care  through  both  the  Biometrics  database  and  CHCS.  The 
samples  are  based  on  data  from  the  Biometrics  database.  Each  record  in  the 
Biometrics  database  is  related  to  inpatient  care  provided  to  an  individual;  the 
care  ended  with  discharge  from  the  reporting  military  treatment  facility  during 
FY  1997. 

We  used  two  samples,  both  drawn  statistically.  The  larger  sample  is  based  on 
6  of  the  116  military  treatment  facilities  and  represents  87,151  of  the  412,049 
discharges  in  FY  1997.  The  second  sample  involves  care  at  only  2  of  the 
6  militap^  treatment  facilities,  representing  36,246  discharges.  Results  from  the 
2  sampling  frames  are  representative  of  only  the  military  treatment  facilities 
from  which  they  were  drawn,  not  of  all  1 16. 

Definition  of  Error  or  Deviation  Condition  (Measurement).  We  have  used 
sample  data  for  two  types  of  testing.  In  one,  the  saniple  records  provide  a 
means  of  examining  the  processes  involved  in  recording  and  measuring  inpatient 
care;  they  are  qualitative  measures  and  are  not  designed  for  statistical 
projection.  The  sample  ensures  a  wide  and  unbiased  selection  of  inpatient 
records  for  testing.  The  second  involves  evaluation  of  a  resource  consumption 
(the  case  weight,  also  referred  to  as  the  relative  weighted  product).  The  case 
weight  is  a  single  number  summarizing  the  relative  quantity  of  resources 
required  to  provide  inpatient  care  for  persons  with  a  common  overall  diagnosis. 
The  case  weight  is  assigned  by  a  coder  based  on  review  of  inpatient  care 
documents  and  stands  as  a  proxy  for  cost  of  care.  We  have  compared  the  case 
weight  as  recorded  with  a  case  weight  determined  by  independent  coders  based 
on  Ae  same  documentary  information  for  a  sample  discharges  at  two  military 
treatment  facilities. 

We  used  our  sample  of  188  records  to  assess  data  and  reporting  processes  at  the 
6  judgraentally  selected  locations  listed  in  the  Appendix  A.  Selecting  the 
discharges  statistically  gave  us  an  unbiased  selection  for  our  testing.  We  used 
the  75  discharges  at  the  David  Grant  and  Portsmouth  medical  centers  to  test 
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consistency  of  case  computed  weight  coding.  Because  the  military  treatment 
facilities  in  each  sample  are  not  representative  of  all  116  military  treatment 
facilities,  our  conclusions  are  limited  to  just  those  military  treatment  facilities 
visited  and  tested. 

Parameters  for  Statistical  Sampling.  We  have  used  sampling  to  assess 
Defense  health  care  information  systems’  reliability  and  completeness  on  a 
limited  basis.  Our  sample  of  coding  differences  used  a  90-percent  confidence 
level.  Because  the  objective  was  to  test  reliability,  versus  sMct  compliance  with 
a  threshold  error  rate,  we  used  two-tailed  testing.  The  aim  is  to  measure 
frequency  and  degree  of  error,  not  to  test  the  sample  data  against  a  standard 
maximum  allowable  error  rate  or  quantity. 

Sampling  Design.  We  used  two  samples  in  our  testing.  We  judgmentally 
selected  6  military  treatment  facilities  and  combined  their  inpatient  care  records 
into  a  single  pool  from  which  we  statistically  selected  a  simple  random  sample 
of  188  records.  We  used  the  records  for  qualitative  testing  of  management 
controls  and  processes.  Because  we  selected  the  military  treatment  facilities 
judgmentally,  the  sample  results  relate  only  to  the  six  military  treatment 
facilities.  In  our  second  sample,  we  selected  a  subset  of  Ae  188  records:  the  75 
that  were  drawn  from  2  military  treatment  facilities  (David  Grant  Air  Force 
Medical  Center  and  Naval  Medical  Center  Portsmouth).  The  75  were  using  a 
simple  random  sample  design;  they  are  statistically  representative  of  the 
36,246  records  from  the  2  military  treatment  facilities  combined.  We  used  the 
Statistical  Analysis  System  programming  language  and  its  random  number 
generators  to  draw  the  sample  of  188  records.  Our  analysis  uses  the  simple 
random  sample  calculation  formula  set  forth  in  Cochran.'  We  used  an  Excel 
spreadsheet  for  our  analysis. 

Results  of  Sampling  of  Re-Coded  Case  Weights.  The  following  results  apply 
statistically  only  to  the  two  military  treatment  facilities  sampled.  In  6  of  the 
75  re-coded  inpatient  discharges,  the  re-coded  DRG  codes  or  the  associated  case 
weights  differed  from  those  recorded.  We  evaluated  the  sample  data  to  address 
the  following  three  issues:  the  frequency  of  changes,  the  average  difference  of 
the  recorded  case  weights  from  the  re-coded  case  weights,  and  the  possibility 
that  the  changes  were  on  a  scale  that  could  have  happened  by  chance. 

Frequency  of  Changes.  Using  a  90-percent  confidence  level,  we 
projected  that  between  3.7  and  15.5  percent  of  the  records  at 
the  two  military  treatment  facilities  would  have  changes.  The  best  single 
estimate  is  2,900,  or  8  percent. 

Recorded  vs.  Re-Coded  Case  Weights.  We  compared  the  case  weights 
reported  on  standard  inpatient  data  records  with  those  resulting  from  the 
re-coding.  The  results  are  summarized  as  follows: 


'  Cochran,  William  G.,  Sampling  Techniques,  3"*  Edition.  New  York.  Wiley,  1977. 
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Case  Weight  Statistical  Bounds 

Mean 

RWP 

Lower 

Bound 

Upper 

Bound 

Case  weight  as 
reported  on  standard 
inpatient  data  record 

0.89 

0.75 

1.04 

Case  weight  as 
re-coded 

0.88 

0.73 

1.02 

We  computed  the  results  using  a  90-percent  confidence  level.  There  was 
no  statistically  significant  difference  between  the  two  sets  of  values. 

Both  the  means  and  their  associated  confidence  intervals  were  very 
nearly  the  same.  The  similarities  are  partly  because  of  the  smallness  of 
the  difference  and  partly  because  of  the  large  amount  of  variation  in  case 
weights.  They  range  from  0.108  to  5.404  in  our  sample.  Given  the 
relatively  small  sample,  along  with  the  very  limited  population 
represented  (only  2  out  of  116  military  treatoent  facilities),  the  sample 
data  showed  no  significant  difference,  but  did  not  prove  there  was  no 
difference  either.  A  conclusion  on  the  overall  inpatient  data  quality  must 
be  based  on  additional  information. 

Differences  From  Changes.  Testing  the  average  change  among  the  75 
sample  inpatient  discharges  yielded  an  average  reduction  in  case  weight 
of  0.012.  Using  a  90-percent  confidence  interval,  the  projected  case 
weight  reduction  ran  from  0.003  to  0.022.  The  conclusion  is  that. 


^  We  tested  the  difference  statistically,  using  the  t-test  for  difference  of  means.  The  resulting  test  value, 
0.09  percent,  did  not  indicate  a  statistically  significant  difference  between  the  two  sets  of  case  weights. 
Sampling  in  this  type  of  analysis  evaluated  the  difference  in  me  ms  and  associated  standard  deviations  and 
answered  the  question  of  whether  the  sample  results  indicate  a  difference  greater  than  would  happen  by 
chance.  We  express  'chance'  in  terms  of  a  confidence  level  -  90-percent  for  the  analyses.  The  changes 
in  case  weights  lead  to  a  small  difference  in  average  weights;  the  difference  was  small  enough  that  we 
cannot  assert,  with  90  percent  confidence,  that  the  average  re-coded  weight  is  different  from  the  CHCS 
weight.  This  could  mean  that  the  re-coded  scores  are  not  significantly  different,  on  the  average,  or  that 
they  are  different,  but  our  sample  did  not  reflect  that  difference.  That  is,  the  sample  results  did  not 
support  a  conclusion  that  the  two  valuation  methods  were  the  same,  only  that  they  were  not  different. 
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with  90  percent  confidence,  there  was  a  reduction  in  case  weights  from 
re-coding  that  did  not  occur  by  chance.^  The  occurrence  indicates  the 
need  for  stronger  management  control  to  ensure  the  accuracy  of  the 
coding  of  inpatient  medical  records. 

The  three  measures  of  re-coding  combined  suggest  the  need  for  further 
work  in  the  area  of  coding.  On  the  one  hand,  the  mean  scores  before 
and  after  re-coding  are  not  significantly  different.  On  the  other,  re¬ 
coding  happened  6  times  among  the  75  items.  Re-coding  reduced  the 
case  weight  or  did  not  change  it.  The  number  of  changes  and  the 
consistency  of  direction  suggest  the  need  for  better  quality  control.  In 
addition,  the  results  are  based  on  only  two  medical  centers,  and  coding 
should  be  tested  across  other  medical  centers  and  community  hospitals. 


^  On  the  other  hand,  the  value  of  the  case  weight  changes  falls  in  the  range  0.003  and  0.022,  at  the 
90-percent  confidence  level,  indicating  that  the  changes  are  greater  than  would  have  happened  by  chance 
90  percent  of  the  time. 
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